The human somatosensory system comprises dissociable paths for discriminative and affective touch, reflected in separate peripheral afferent populations and distinct cortical targets. Differences in behavioral and neural responses to affective touch may have an important developmental role in early social experiences, which are relevant for autism spectrum disorder (ASD). Using probabilistic tractography, we compared the structural integrity of white matter pathways for discriminative and affective touch in young children with ASD and their typically developing (TD) peers. We examined two tracts: (1) a tract linking the thalamus with the primary somatosensory cortex, which carries discriminative tactile information, and (2) a tract linking the posterior insula—the cortical projection target of unmyelinated tactile afferents mediating affective touch—with the anterior insula, which integrates sensory and visceral inputs to interpret emotional salience of sensory stimuli. We investigated associations between tract integrity and performance on a standardized observational assessment measuring tactile discrimination and affective responses to touch. Both the thalamocortical and intrainsular tracts showed reduced integrity (higher mean diffusivity) in the ASD group compared to those in the TD group. Consistent with the previous findings, the ASD group exhibited impaired tactile discriminative ability, more tactile defensiveness, and more sensory seeking (e.g., enthusiastic play or repetitive engagement with a specific tactile stimulus). There was a significant relation between intrainsular tract integrity and tactile seeking. The direction of this relation differed between groups: higher intrainsular mean diffusivity (MD) (reflecting decreased tract integrity) was associated with increased tactile seeking in the TD group but with decreased tactile seeking in the ASD group. In the TD group, decreased tactile defensiveness was also associated with higher intrainsular MD, but there was no relation in the ASD group. Discriminative touch was not significantly associated with integrity of either tract in either group. These results support previous findings suggesting a central role for the insula in affective response to touch. While both discriminative and affective touch and both somatosensory tracts are affected in ASD, the restriction of brain–behavior associations to the intrainsular tract and tactile seeking suggests more complex and perhaps higher-order influence on differences in tactile defensiveness and discrimination.